A major trend in illumination is solid-state lighting that uses light emitting diodes (LEDs). As LEDs begin to replace incandescent and fluorescent lamps, LEDs can be used for multiplexed optical communication with intelligent devices in addition to providing illumination. Solid-state optical emitters are compact, can be modulated at a high data rate, and can be selected to emit light as a narrow bandwidth optical signal.
It is widely accepted that solid-state lights will soon be ubiquitous—in homes, offices and shops, Tsao, “Light emitting diodes (LEDs) for general illumination,” U.S. Department of Energy Lighting Technology Roadmap, 2002; and Talbot, “LEDs vs. the lightbulb,” MIT Technology Review, 2003.
Conventional scene acquisition methods use cameras to determine an interaction between geometric and photometric attributes of a scene. However, analyzing scenes using camera images is known to be a difficult inverse light transport problem, because the radiance measured at each camera pixel is a complex function of geometry, illumination, and reflectance.
Optical Communication and Demodulation
It is now possible to achieve several modulation operations on optical signals that were once only possible in the radio frequency (RF) range. Among the motivations for selecting optical communication instead of radio frequency communication are benefits such as directionality, lack of interference in RF sensitive environments, and higher bandwidths.
While a majority of the prior art solid state lighting applications are in communication for point-to-point data transfer, these concepts can be extended to free-space interaction. A remote control for a device is an example where an infrared LED is temporally modulated by binary codes at a carrier frequency of about 40 KHz. The signal is acquired by a photo sensor mounted at the front of the device and demodulated to perform various device functions.
Other systems allow incandescent lights to communicate with devices in a room, Komine and Nakagawa, “Fundamental analysis for visible-light communication system using LED lights,” IEEE Transactions on Consumer Electronics, vol. 50, no. 1, pp. 100-107, 2004. Vehicle tail-lights can continuously transmit speed and braking conditions in a narrow direction to following vehicles, Misener et al., “Sensor-friendly freeways: Investigation of progressive roadway changes to facilitate deployment of AHS,” Tech. Rep. UCB-ITS-PRR-2001-31, 2001.
Location Tracking
Several techniques for motion tracking using magnetic, acoustic, optical, inertial, or RF signals are available, Welch and Foxlin, “Motion tracking: No silver bullet, but a respectable arsenal,” IEEE Comput. Graph. Appl., vol. 22, no. 6, pp. 24-38, 2002.
Typically, optical systems have shorter latencies and provide greater accuracy. In addition, an optical channel can be exploited to investigate the photometric aspect of scene capture.
Most motion capture systems used in movie studios employ high-speed cameras to observe passive visible markers or active LED markers. For example, a high-speed digital video camera can record 1280×1024 full-frame grayscale pixels at speeds of up to 484 frames per second with onboard processing to detect the marker position. Those devices provide highly reliable output data. However, the extremely expensive high-speed cameras pose several issues in terms of scalability. Bandwidth limits the resolution as well as the frame-rate. Higher frame-rates demand shorter exposure time. That requires bright controlled scene lights for the passive markers or the use of battery operated LED markers. To segment the markers from the background, those systems also use methods for increasing marker contrast. That usually requires that the actor wears dark clothes in a controlled lighting situation. It is desired to perform motion capture in natural settings.
Photo Sensing
A number of systems are known for locating objects having attached photo sensors, Ringwald, “Spontaneous Interaction with Everyday Devices Using a PDA,” Workshop on Supporting Spontaneous Interaction in Ubiquitous Computing Settings, UbiComp, 2002; Patel and Abowd, “A 2-Way Laser-Assisted Selection Scheme for Handhelds in a Physical Environment,” UbiComp, pp. 200-207, 2003; and Ma and Paradiso, “The FindIT Flashlight: Responsive Tagging Based on Optically Triggered Microprocessor Wakeup,” UbiComp, pp. 160-167, 2002.
Other systems locate photo sensing RFID tags with a conventional digital projector, Nii et al., “Smart light ultra high speed projector for spatial multiplexing optical transmission,” International Workshop on Projector-Camera Systems, Jun. 25, 2005, San Diego, Calif., USA; Raskar et al., “RFIG lamps: Interacting with a self-describing world via photosensing wireless tags and projectors,” ACM Transactions on Graphics vol. 23, no. 3, pp. 406-415, 2004; Lee et al., “Moveable interactive projected displays using projector based tracking,” UIST 2005: Proceedings of the 18th annual ACM symposium on user interface software and technology, ACM Press, New York, N.Y., USA, pp. 63-72, 2005; and U.S. patent application Ser. No. 10/643,614 “Radio and Optical Identification Tags” filed by Raskar on Aug. 19, 2003, Ser. No. 10/883,235 “Interactive Wireless Tag Location and Identification System” filed by Raskar et al. on Jul. 1, 2004, and Ser. No. 10/030,607 “Radio and Optical Identification Tags” filed by Raskar on Jan. 5, 2005 all incorporated herein by reference.
The UNC “HiBall” system uses a group of six rigidly fixed position sensitive detectors (PSD) to find location and orientation with respect to actively blinking LEDs, see Welch, “Scaat: Incremental tracking with incomplete information,” UNC Tech. Report, Chapel Hill, N.C., USA, 1886. Each LED provides a single under-constrained reading at a time. That system requires a large ceiling installation, and active control of the LEDs operating in an open loop.
Systems such as “Indoor GPS” use low-cost photo sensors and two or more rotating light sources mounted in the environment, Kang and Tesar, “Indoor GPS metrology system with 3d probe for precision applications, Kang, S. and Tesar, D., “A Noble 6-DOF Measurement Tool With Indoor GPS For Metrology and Calibration of Modular Reconfigurable Robots,” IEEE ICM International Conference on Mechatronics, Istanbul, Turkey, 2004. The rotating light sources sweep out distinct planes of light that periodically illuminate the photo sensors. That system operates at a rate of 60 Hz.
Factoring Reflectance and Illumination
Scene Factorization
Scene factorization, as defined herein, is a computer vision technique for inferring scene parameters. The scene parameters can include geometry and photometry. The geometry defines the 3D locations, orientations, and shapes of objects in the scene, and the photometry defines the interaction of light with the objects. The light can be due to direct illumination, reflectance, radiance, and translucency, see generally, C. Tomasi and T. Kanade, “Shape and Motion from Image Streams: A Factorization Method,” Proceedings of the National Academy of Sciences, vol. 90, pp. 995-9802, 1993.
Camera-based factorization of scene radiance into a product of incident illumination and albedo is known in computer vision applications. The problem formulation can involve multiple views, a single view with variable illumination, or both, Forsyth and Ponce, “Computer Vision, A Modern Approach,” 2002.
However, scene factorization is an ill-posed problem and solutions require assumptions regarding the reflectance variation in the scene and/or the illumination variation due to the light source.
Communication with Optical Tags
The use of spatio-temporal optical modulation is influenced by developments in radio frequency, available bandwidth in optical communication, and opportunities in projective geometry. In light based communication, the optical bandwidth, which is a product of temporal and spatial bandwidth, has been increasing annually by about a factor of three. The penetration of solid state LEDs in diverse fields, such as optical networking, CD readers, and IrDA, indicates a trend in versatile temporal modulation of light sources. At the same time, high resolution spatial modulation is becoming possible via microelectromechanical (MEMS) based, liquid crystal on silicon (LCOS), grating light valve (GLV) and traditional liquid crystal display (LCD) imagers.
Optical Communication Tools
A range of optical emitter and receiver devices are available for use in an optical scene capture systems. Typically, there is a tradeoff in complexity of the emitter, the receiver and bandwidth. It is difficult to achieve high-speed scene capture with low-cost, simple devices.
It is well known that a limited dynamic range is best utilized through time-division multiplexing, followed by frequency- and code-division multiplexing (FDM and CDM), Azizoglu et al., “Optical cdma via temporal codes,” IEEE Transactions on Communications, vol. 40, no. 7, pp. 1162-1170, 1992.
Therefore, it is desired to factorize scenes using low-cost solid state light emitters and sensors.